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DETAILED ACTION 

1 . This action is in response to the application filed 3/29/2006. 

2. Claims 1-24 have been submitted for examination. 



Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

3. Claims 1, 4-8, 10-13, 16-19, and 21-24 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Wilson et al. (US 2004/0193413), hereinafter Wilson. 



4. As per claims 1 , 12, 23 and 24, Wilson teaches the following: 

a display surface (108, 300) for displaying content to a user, (pg. 5, paragraph 
[0060]), i.e. the system comprises a computer 200 with a traditional keyboard 202, input 
pointing device 204, microphone 206, and display 208 ; 

a computer system (110) for supplying the content to the display surface (108) 
for display in a content window (1 12, 306) on the display surface (108, 300), (pg. 5, 
paragraph [0060]), i.e. the system comprises a computer 200 with a traditional keyboard 
202, input pointing device 204, microphone 206, and display 208, (pg. 8, paragraph 



Application/Control Number: Page 3 

10/574,137 

Art Unit: 2176 

[0083]), i.e. typical GUI windows 712, 714, and 716 are displayed in a layered 
presentation in an upper portion of display 718 : and 

a recognition system ( 128) for recognizing a gesture of a user and defining at 
least one of a size, location, and content of the content window (112, 306) on the 
display surface (108) based on the recognized gesture, (pg. 8, paragraph [0088]), i.e. 
Fig. 7 illustrates the embodiment where a single hand shaped icon 720 is used, and the 
corresponding command recognized by the system is displayed below the icon 720. 
For example, when the system recognizes , either by virtue of gestures (with hand 
and/or object) and or verbal commands, the command to move a window , the icon 720 
and corresponding command word "MOVE" are displayed by the display 718. 

5. Regarding claims 4 and 16, Wilson teaches the method of claims 1 and 12 as 
described above. Wilson further teaches the following: 

one or more sensors operatively connected to the computer system ( 1 10), (pg. 5, 
paragraph [0060]), i.e. video cameras 210 and microphone 206 are also operably 
connected to computer 200 : 

* 

a processor (1 14) for analyzing data from the one or more sensors to 
recognize the gesture of the user, (pg. 5, paragraph [0060]), i.e. the hand gestures 
and/or object(s) appear in video images created by the video cameras 210 and are 
interpreted by the software 214 as commands to be executed by computer 200. 

The examiner would like to make further note of Wilson's teaching on pg. 4, 
paragraph [0051], where Wilson directly states that a "component" or "system" may be 
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interpreted to be a process running on a processor or a processor . 

6. Regarding claims 5 and 17, Wilson teaches the method of claims 4 and 16 as 
described above. Wilson further teaches the following: 

the one or more sensors comprise one or more cameras (130), (pg. 5, 
paragraph [0060]), i.e. video cameras 210 and microphone 206 are also operably 
connected to computer 200; 

wherein the processor analyzes image data from the one or more cameras (130) 
to recognize the gesture of the user, (pg. 5, paragraph [0060]), i.e . the hand gestures 
and/or object(s) appear in video images created by the video cameras 210 and are 
interpreted by the software 214 as commands to be executed by computer 200. 

7. Regarding claims 6 and 18, Wilson teaches the display of claims 4 and 16 as 
described above. Wilson further teaches the following: 

the recognition system (128) further comprises a memory (116) for storing 
predetermined gestures and an associated size and/or position of the content window 
(1 12, 306), wherein the processor (1 14) further compares the recognized gesture of the 
user to the predetermined gestures and renders the content window ( 1 12) in the 
associated size and/or position, (pg. 10, paragraph [0102]), i.e. in the "Record and 
Define Gesture" mode, the system records hand gestures performed by the user. The 
recorded gestures are then stored in the system memory to be recognized during 
normal operation. The given hand gestures are then associated with a particular 
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command to be performed by the system in response to that particular hand gesture, 
(pg. 8, paragraph [0088]), i.e. Fig. 7 illustrates the embodiment where a single hand 
shaped icon 720 is used, and the corresponding command recognized by the system is 
displayed below the icon 720. For example, when the system recognizes , either by 
virtue of gestures (with hand and/or object) and or verbal commands, the command to 
move a window , the icon 720 and corresponding command word "MOVE" are displayed 
by the display 718. 

8. Regarding claims 7 and 19, Wilson teaches the display of claims 6 and 18 as 
described above. Wilson further teaches the following: 

the memory (116) further includes an associated content, wherein the processor 
(114) further compares the recognized gesture of the user to the predetermined 
gestures and renders the associated content in the content window, (112, 306), (pg. 8, 
paragraph [0086]), i.e. here, the same hand icon 720 is displayed in accordance with 
four different hand gestures utilized to indicate four different commands: Move, Close, 
Raise, and Scroll . 

The examiner interprets Wilson's teaching of "scrolling" the content of a window 
to encompass applicant's limitation of rendering associated content. 

9. Regarding claim 8, Wilson teaches the display of claim 6 as described above. 
Wilson further teaches the following: 
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wherein the processor (114) and memory (116) are contained in the computer 
system (110). As Wilson teaches in pg. 4, paragraph [0051], a "component" and 
"system" may be interpreted to be a process running on a processor and a "component" 
may be localized in one computer . Furthermore, as Wilson teaches in pg. 10, 
paragraph [0102] that hand gestures are stored in a system memory , it would have 
been clear that the memory and processor may exist within the computer system. 

10. Regarding claims 10 and 21, Wilson teaches the method of claims 1 and 12 as 

* 

described above. Wilson further teaches the following: 

the gesture further defines a closing of an application displayed on the display 
surface (108, 300), (pg. 8, paragraph [0086]), i.e. here, the same hand icon 720 is 
displayed in accordance with four different hand gestures utilized to indicate four 
different commands: Move, Close . Raise, and Scroll. 

1 1 . Regarding claims 1 1 and 22, Wilson teaches the method of claims 1 and 12 as 
described above. Wilson further teaches the following: 

one of a touch-screen, close-touch, andtouchless system (122, 124, 126) for 
entering a command into the computer system, (abstract), i.e. a 3-D imaging system for 
recognition and interpretation of gestures to control a computer. 

The examiner interprets Wilson's method of "recognizing and interpreting 
gestures" to be that of a "touchless" system. 
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The examiner would like to further note Wilson's teaching on pg. 4, paragraph 
[0057] where Wilson explicitly teaches that their system may receive input via a touch 
screen. 

12. Regarding claim 13, Wilson teaches the method of claim 12 as described above. 
Wilson further teaches the following: 

the gesture is a hand gesture, (pg. 8, paragraph [0088]), i.e. Fig. 7 illustrates the 
embodiment where a single hand shaped icon 720 is used, and the corresponding 
command recognized by the system is displayed below the icon 720. For example, 
when the system recognizes, either bv virtue of gestures (with hand and/or object) and 
or verbal commands, the command to move a window, the icon 720 and corresponding 
command word "MOVE" are displayed by the display 718. 

Claim Rejections - 35 USC § 103 

13. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

14. Claims 2, 3, 14 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wilson as applied to claims 1 and 12 as described above, in view of 
Taguchi (US 5,793,367). 
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15. Regarding claims 2 and 15, Wilson teaches the method of claims 1 and 12 as 
described above. However, Wilson does not explicitly teach a method where the 
display is a display mirror. Taguchi teaches the following: 

the display is a display mirror for reflecting an image of the user at least when the 
content is not being displayed, (column 3, lines 37-39), i.e. in an embodiment of the 
present invention, a moving image photographed by a camera is displayed on a window 
in an image display window system as a dedicated window for controlling the camera. 
The examiner interprets Taguchi's method of displaying what a camera is 
photographing to encompass applicant's limitation of "a display mirror". 

Moreover, as admitted by Applicant in the Specification of the present invention 
(see Page 1, Lines 6-9), the prior art discloses a display mirror for reflecting an image of 
the user at least when the content is not being displayed in that a mirror may also 
include a content window that displays information on a particular part of the mirror. 

It would have been obvious to one of ordinary skill in the art to have modified the 
gesture based control method of Wilson with the moving image display of Taguchi. One 
skilled in the art would have been motivated to have made such modifications because 
Wilson is directed towards a method of controlling a graphical user interface, in 
particular, a window based interface (see pg, 1, paragraph [0007]). Wilson does not 
limit the content of the windows in the interface and purely discusses a control method 
of the windows. Taguchi teaches of a method of displaying, in a window , a moving 
image of data picked up by an image pickup device (see abstract). Therefore, it would 
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have been obvious to have modified one of the windows of Wilson's graphical interface 
to be that of the picked up moving image of Taguchi. Further motivation for such 
modifications may be found in the similar architecture of the two systems. As may be 
seen in Taguchi's showing of Fig. 2, their system includes a computer system 30, a 
display 32, and a motion camera 40. Wilson has a similar system as may bee seen in 
their showing of Fig. 2, where their system includes a computer 200, a display 208, and 
motion cameras 210. Thus Wilson's system possesses all of the hardware necessary to 
support the display method of Taguchi. 

With respect to applicant's limitation of displaying the mirror reflection image 
when at least the content is not being displayed or only displaying the content, these 
limitations are encompassed by Wilson's teaching of "maximizing" a window (see pg. 
10, paragraph [0098]). As was well known in the art, upon a window becoming 
"maximized", that window encompasses substantially the entire display area, hiding 

4 

* 

other content windows. Therefore, upon a user of the modified system of Wilson 
"maximizing" the window containing the mirror image of Taguchi, that mirror image 
window would essentially encompass substantially the entire display, therefor hiding the 
content windows. Furthermore, upon a user of the modified system of Wilson 
"maximizing" a content window other than the one containing the mirror image of 
Taguchi, that content window would essentially encompass substantially the entire 
display, therefor hiding the mirror window. 
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16. Regarding claims 3 and 14, modified Wilson teaches the method of claims 2 and 
12 as described above. Wilson further teaches the following: 

the display mirror displays both the content and the image of the user, (pg. 8, 
paragraph [0083]), i.e. typical GUI windows 712, 714. and 716 are displayed . 

This teaching of Wilson encompasses applicant's limitation of displaying an 
image of the user and the content in that as Wilson teaches that multiple windows are 
displayed, the modified system of Wilson in view of Taguchi may have the mirror image 
of Taguchi in one window and other content in another, thus displaying both a mirror 
image and a content window. 

Moreover, as admitted by Applicant in the Specification of the present invention 
(see Page 1, Lines 6-9), the prior art discloses a display mirror [that] displays both the 
content and the image of the user in that a mirror may also include a content window 
that displays information on a particular part of the mirror. Thus, while the content 
window displays information, other portions of the mirror displays reflections of a user. 

17. Claims 9 and 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Wilson as applied to claims 1, 4, and 12 as described above, in view of Sun (US 
6,643,721). 
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18. Regarding claims 9 and 20, Wilson teaches the method of claims 1 and 12 as 
described above. However, Wilson does not explicitly teach a method where a speech 
command renders content in the window. Sun teaches the following: 

a speech recognition system (132) for recognizing a speech command of the 
user and rendering a content in the content window ( 1 12, 306) based on the recognized 
speech command, (column 8, lines 8-10), Fig. 5b shows the same graphical interface 
element as Fig. 5a, as it appears when the primary input device is a voice recognition 
input , (column 8, lines 13-16), i.e. the appearance resources adapt to the new primary 
input device by assigning a simple letter name to the scrollbar (Scroll A), and providing 
other word cues for possible user commands (Top, Bottom, Up, Down). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have modified the "scrolling" method of Wilson with verbal 
commands of Sun. One skilled in the art would have been motivated to have made 
such modifications because both Wilson and Sun are analogous art in the field of 
controlling windows based graphical user interfaces using gestures and voice 
commands. Furthermore, Wilson directly teaches in pg. 8, paragraph [0085] that "a 
Scroll command may be initiated first by voicing a corresponding command that is 
processed by speech recognition" thus showing the desire to involve verbal commands 
with the scrolling process. 
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Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

-Freeman (US 5,454,043), dynamic and static hand gesture recognition through 
low-level image analysis. 

-Sagawa et al. (US 5,734,923), apparatus for interactively editing and outputting 
sign language information using graphical user interface. 

-Reichlen (US 6,061,064), system and method for providing and using a 
computer user interface with a view space having discrete portions. 

-Nguyen (US 6,072,494), method and apparatus for real-time gesture 
recognition. 

-Cox et al. (US 6,154,723), virtual reality 3D interface system for data creating, 
viewing and editing. 

-Kumar et al. (US 6,222,465), gesture-based computer interface. 

-Bradski (US 6,394,557), method and apparatus for tracking an object using a 
continuously adapting mean shift. 

-Cohen et al. (US 6,681,031), gesture-controlled interface for self-service 
machines and other applications. 

-Pryor et al. (US 6,720,949), man machine interfaces and applications. 

-Bermudez et al. (US 6,895,589), manager component for managing input from 
existing serial devices and added serial and non-serial devices in a similar manner. 
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-Wilson (US 6,990,639), system and process for controlling electronic 
components in a ubiquitous computing environment using multimodal integration. 

-Krahnstoever et al. (US 6,996,460), method and apparatus for providing virtual 
touch interaction in the drive-thru. 

-Inagaki et al. (US 7,046,232), information processing apparatus, method of 
displaying movement recognizable standby state, method of showing recognizable 
movement, method of displaying movement recognizing process, and program storage 
medium. 

-Mattsson (US 2003/0076293), gesture recognition system. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gregory A. DiStefano whose telephone number is 
(571 )270-1644. The examiner can normally be reached on 7:30am-5:00pm Mon.- 
Thurs.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doug Hutton can be reached on (571 )272-4137. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

GAD 

1/4/2008. 

/(Doug ffutton/ 

Doug Hutton 
Supervisory Primary Examiner 
Technology Center 2100 



